Survey Methodology: International Developments

Falling response rates and the advancement of technology have shaped the discussion in survey methodology in the last few years. Both led to a notable change in data collection efforts. Survey organizations try to create adaptive recruitment and survey designs and increased the collection of non-survey data for sampled cases. While the first strategy is an attempt to increase response rates and to save cost, the latter is part of efforts to reduce possible bias and response burden of those interviewed. To successfully implement adaptive designs and alternative data collection efforts researchers need to understand error properties of mixedmode and multiple-frame surveys. Randomized experiments might be needed to gain that knowledge. In addition close collaboration between survey organizations and researchers is needed, including the possibility and willingness to shared data between those organizations. Expanding options for graduate and post-graduate education in survey methodology might help to increase the possibility for high quality surveys.Survey Methodology, Responsive Design, Paradata

Survey methodology: international developments

"Falling response rates and the advancement of technology have shaped the discussion in survey methodology in the last few years. Both led to a notable change in data collection efforts. Survey organizations try to create adaptive recruitment and survey designs and increased the collection of non-survey data for sampled cases. While the first strategy is an attempt to increase response rates and to save cost, the latter is part of efforts to reduce possible bias and response burden of those interviewed. To successfully implement adaptive designs and alternative data collection efforts researchers need to understand error properties of mixedmode and multiple-frame surveys. Randomized experiments might be needed to gain that knowledge. In addition close collaboration between survey organizations and researchers is needed, including the possibility and willingness to shared data between those organizations. Expanding options for graduate and post-graduate education in survey methodology might help to increase the possibility for high quality surveys." [author's abstract

Paradata

Paradata – data about the process of survey production – have drawn increasing attention as the statistical world moves towards the implementation of quality metrics and measures to improve quality and save costs. This paper gives examples of various uses of paradata and discusses access to paradata as well as future developments.paradata, process data, responsive design, measurement error, nonresponse, adjustment

Separating interviewer and sampling-point effects

"Data used in nationwide face-to-face surveys are almost always collected in multistage cluster samples. The relative homogeneity of the clusters selected in this way can lead to design effects at the sampling stage. Interviewers can further homogenize answers within the small geographic clusters that form the sampling points. The study presented here was designed to distinguish between interviewer effects and sampling-point effects using interpenetrated samples for conducting a nationwide survey on fear of crime. Even though one might, given the homogeneity of neighborhoods, assume that sampling-point effects would be especially strong for questions related to fear of crime in one's neighborhood, we found that, for most items, the interviewer was responsible for a greater share of the homogenizing effect than was the spatial clustering. This result can be understood if we recognize that these questions are part of a larger class of survey questions whose subject matter is either unfamiliar to the respondent or otherwise not well anchored in the mind of the respondent. These questions permit differing interpretations to be elicited by the interviewer." (author's abstract

Multiple Auxiliary Variables in Nonresponse Adjustment

Prior work has shown that effective survey nonresponse adjustment variables should be highly correlated with both the propensity to respond to a survey and the survey variables of interest. In practice, propensity models are often used for nonresponse adjustment with multiple auxiliary variables as predictors. These auxiliary variables may be positively or negatively associated with survey participation, they may be correlated with each other, and can have positive or negative relationships with the survey variables. Yet the consequences for nonresponse adjustment of these conditions are not known to survey practitioners. Simulations are used here to examine the effects of multiple auxiliary variables with opposite relationships with survey participation and the survey variables. The results show that bias and mean square error of adjusted respondent means are substantially different when the predictors have relationships of the same directions compared to when they have opposite directions with either propensity or the survey variables. Implications for nonresponse adjustment and responsive designs will be discussed

Using paradata to explore item level response times in surveys

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/95020/1/rssa1041.pd

A Survey on Survey Statistics: What is done, can be done in Stata, and what's missing

Among survey statisticians Stata is increasingly recognized as one of the more powerful statistical software packages for the analysis of complex survey data. This paper will survey the capabilities of Stata to analyze complex survey data. We will briefly review and compare different methods for variance estimation for stratified and clustered samples, and discuss the handling of survey weights. Examples will be given for the practical importance of Stata's survey capabilities. In addition we will point to statistical solutions that aren't yet part of the official package, and review user written ados that currently extend Stata's survey capabilities. Among the specific topics we will cover are replication variance estimation (jackknife, balanced repeated replication, and the bootstrap), issues associated with degrees of freedom and domain estimates, quantile estimation, and some concerns related to model fitting using survey data.